There exist numerous applications in which real time data analysis may be required. For example, data events may be collected in a financial setting to identify potentially fraudulent activity, in a network setting to track network usage, in a business setting to identify business opportunities or problems, etc. Often, it may be necessary to examine individual data events as they occur to immediately investigate any suspect behavior. Challenges however arise when analyzing data events in real time since historical data values are typically necessary to identify trends and patterns. Namely, accessing historical data can be a relatively slow process, and thus limits real time processing.
There exist various known techniques (e.g., running estimates, moving windows, etc.) for analyzing data events in real time (or near real time). In such techniques, the historical data is essentially “built in” to the currently calculated estimate, thus providing a statistical summary in a single value (or set of values). Such techniques utilize little or no historical data to provide a statistical analysis of detected event values. Instead, they, e.g., maintain a running value, which is updated each time a new data event value is collected.
In some applications, it may be desirable to identify an overall trend in a stream of data events as they occur in real time. Based on such identification, a first level alert of a trend could be issued and responded to in an appropriate manner. However, because real time analysis techniques do not have the luxury of examining significant amounts of historical data, current techniques lack an effective ability to identify overall trends. Accordingly, a need exists for a system and method of detecting trends in a real time data analysis setting.